Description

For one of my projects, I needed to convert a trained BERT model, which was only available on my local filesystem. However, when I attempted the conversion, I encountered an error indicating that my tokenizer was not supported. To resolve this, I added my local model and its filesystem path to the models array, but running the convert-hf-to-gguf-update.py script did not update convert-hf-to-gguf.py as expected.

The root cause was identified in the download_model function. The current implementation only supports downloading models hosted on HuggingFace.

But my fix extends this functionality to also support models stored locally.

Changes Made

• Modified the download_model function to check if the repo path is a local directory.
• If repo is a local path, the function now copies the required files from the local directory to the appropriate destination.
• If repo is a URL, the function retains its original behavior of downloading the files from HuggingFace.

Example Usage

To convert a local model, you can now add it to the models array of convert-hf-to-gguf-update.py with the local filesystem path:

models = [
    {"name": "sbert", "tokt": TOKENIZER_TYPE.WPM, "repo": "./sbert_large_nlu_ru"},
    ...
    ...
]

Run it:

python convert-hf-to-gguf-update.py <hf_token>

And new block will be added to convert-hf-to-gguf.py, it will looks like this:

if chkhsh == "eb70cf7b9840b5efff6c673f695000d9a0195852c07a45a3e61718fcbaddf601":
    # ref: ./sbert_large_nlu_ru
    res = "sbert"

After than you will be able to convert local model to the GGUF format.